Chapter 2

This chapter begins with an introduction to Artificial Intelligence (AI) to discuss the first things that most AI learners want to know: why look into AI, what is AI, the history of AI, AI vs. traditional engineering methods, and AI applications in engineering and other sectors. Next, the basics of AI are laid out, including the basic concepts, common algorithms, and challenges and issues.

Mathematics takes different roles in different AI topics and is thus essential in the understanding and implementation of many of them. Considering the possibly diverse backgrounds of the readers, such knowledge is provided systematically at the end of the book as Appendices. Mathematics knowledge that is needed for AI, i.e., statistics, information theory, and array operations, can be reviewed there as needed. Some math knowledge that is essential to specific AI topics will be provided in the corresponding chapters.

Deemed the core of the fourth industrial revolution or Industry 4.0, AI has been reshaping our lives in many ways towards what we have long pictured in science fiction. From chatbots like ChatGPT to autonomous cars, more widespread AI applications have been transforming our society, bringing benefits such as increased efficiencies, more intelligent products, and fewer repetitive tasks. AI is projected to boost the corporate profitability of 16 industries in 12 economies by an average of $38\mathrm{\%}$$38\mathrm{\%}$38%38 \%$38\mathrm{\%}$ by 2035 [1].

The popularity and impacts of AI can be seen in related intellectual products. First, the number of AI-related publications almost tripled in one decade, e.g., from 88,000 in 2010 to 240,000 in 2022 in AI categories like pattern recognition, machine learning, and algorithms [2, 3]. Besides publications, the number of AI-related patent filings multiplied by a factor of 30 times from 2015 to 2021. The global market size of AI software is expected to grow at a rate of $22.97\mathrm{\%}$$22.97\mathrm{\%}$22.97%22.97 \%$22.97\mathrm{\%}$ from 2023 and surpass $\mathrm{\$}1,094.52$$\mathrm{\$}1,094.52$$1,094.52\$ 1,094.52$\mathrm{\$}1,094.52$ billion by 2032 [4].

This so-called third wave/tide of AI has enabled world-changing applications represented by deep learning and reinforcement learning. Deep learning gained success in application areas like computer vision and natural language processing. Computer vision is an AI subfield that teaches machines to understand images and videos for the purposes of image classification, object detection, mapping of position and movement of objects, and so on. In addition to understanding existing images, AI can also generate "fake" images and videos that are nearly indistinguishable from real ones. Natural language processing, a subfield that focuses on summarizing contents, inferring outcomes, identifying emotional context, speech recognition and transcription, and translation, is impacting our lives again via many language-related innovations and tools, attributed to its cumulative development since the 1950s and recent breakthroughs via deep learning especially large language models. Besides deep learning, reinforcement learning is another major area of $AI$$AI$AIA I$AI$ innovations, which helps us advance the cutting edge of AI. Its triumphs have swept from video games to complicated board games like chess and Go, and more recently, in engineering decision-making and control tasks like robotics.

AI grows and impacts the world via strong synergy with the blooming of data and improvements in computing hardware. The third wave of AI is propelled by big data, improvements in computing software (GPU), and advances in machine learning algorithms. In engineering, the explosion of data is partly attributed to the widespread use of low-cost
information-sensing mobile devices, remote sensing, software logs, cameras, microphones, radio-frequency identification (RFID) readers, and wireless sensor networks. The vast amounts of data are enabling us to explore AI tools with more complicated architectures, higher capacity, and better generalization, such as neural networks that are deeper, wider, and more intricate inside. In return, AI breakthroughs driven by data promoted tech giants like Google, Microsoft, Meta, IBM, and Amazon to improve AI tools, algorithms, applications, and data. Owning to the development of AI, a system that would have taken 6 minutes to train in 2018 would only need about 13 seconds in 2021 [2]. In terms of cost, The costs of AI training drop at a rate of $70\mathrm{\%}$$70\mathrm{\%}$70%70 \%$70\mathrm{\%}$ per year as of 2024 [5], outpacing the famous Moore's Law.

The dramatic development of AI has also been obviously affecting the future of the workforce. More AI applications across industries create higher demands for AI education and jobs. Taking the U.S., for example, California, Texas, New York, and Virginia have exhibited high demands for AI-related occupations. In computer science, the most popular specialties among Ph.D. students have been secured by AI and machine learning in the past decade. In addition to the trends in the workforce demand and development, corporate investments in AI are at an all-time high, totaling $189 billion in 2023, representing a $1300\mathrm{\%}$$1300\mathrm{\%}$1300%1300 \%$1300\mathrm{\%}$ increase from 2013, and could amount to $\mathrm{\$}200$$\mathrm{\$}200$$200\$ 200$\mathrm{\$}200$ billion globally by 2025 [3]. AI companies in the "healthcare" sector led the champion, followed by "data management, processing, and cloud" and financial technology (or called "fintech for short").

AI also generates far-reaching impacts on the economy, politics, mobility, healthcare, security, and the environment. Such influence on the economy can take place via disruptions to the labor market, alternation of the nature of longestablished roles, and changes in political thinking and opinions. For mobility, AI is estimated to be capable of helping reduce the number of road accidents by as much as $90\mathrm{\%}$$90\mathrm{\%}$90%90 \%$90\mathrm{\%}$ and boosting multimodal transportation through better transportation options and operations, while also brings about new challenges in liability, ethics, and management [6]. The further development of AI for healthcare can possibly eradicate many incurable diseases and help deliver care to remote areas and groups with difficulties. In security and defense sectors, AI-powered software is dramatically altering the digital security threat landscape via innovative cyber attack detection, prevention, and risk control, which could easily save economic loss in excess of $\mathrm{\$}50$$\mathrm{\$}50$$50\$ 50$\mathrm{\$}50$ billion in one major global cyber-attack [7].

Therefore, AI is a must-know for the new generations. For engineers, we may need to know the basics as engineering is being further impacted by and fused with AI. Some engineers who will be more exposed to or deal with AI may need to know the common AI techniques, from the entry level of having common and useful AI techniques in their toolbox to a more advanced level of assessing, modifying, extending, and coding some newer and complicated AI algorithms. This book is proposed to help engineers quickly bridge these gaps.

AI has been defined from different perspectives by people from many distinct areas and thus encompasses a wide variety of techniques. In this book, AI is defined as a method or the use of such a method for making a computer or a computercontrolled agent, either hardware like a robot or software like a computer program, to think intelligently like the human mind. Thus, AI is accomplished by studying the working mechanisms of human brains and by analyzing the cognitive processes. Such AI studies develop products like data, algorithms, intelligent software and systems, and paradigms for specific applications. All of such AI products enable the computer or any agent controlled by it to exhibit some types of human beings' intelligence to some extent.

AI can be classified in different ways, which is a topic that can trigger many inconsistencies, conflicts, and debates. This fact is attributed to numerous reasons. In particular, as an area contributed by people from different disciplines, the history, convention, and backgrounds of these contributors in AI could lead to the adoption of the same terms for different meanings, names, and usages and different terms with the same meaning. In addition, the evolution of the AI field including the technical development and other incidents such as re-branding of AI topics for promoting publications and fundraising further created gaps, overlaps, conflicts, and confusion when talking about the types of AI as well as the categorization of its subareas.

One classification constructed based on the consensuses in the literature will be adopted throughout this book for consistency. As shown in Fig. 2.1, AI can be roughly categorized into general AI and narrow AI. General AI is what we see in science fiction, in which AI can enable intelligent agents like supermen, e.g., Terminator and Wall-E, for handling much different tasks like combating and flying. Though general AI is always a dream and has been repeatedly discussed in the history of AI including the recent artificial general intelligence, it may still be far from us. By contrast, narrow AI is what we have been mostly working on. Narrow AI is set for a lower goal and thus more feasible. More interestingly, the successes in narrow AI in different phases of the AI history promoted people to pursue general AI.

Narrow AI can be divided into two groups: symbolic AI and non-symbolic AI (or called numeric AI less frequently).

Figure 2.1: Subareas in AI

Symbolic AI is represented by logic reasoning that was extensively studied in the early stages of AI and by more successful applications in expert systems. Non-symbolic AI mostly refers to machine learning. What differentiates these two major AI streams is not the use of symbols or languages that can be easily understood by human beings. Although symbolic AI is usually associated with the use of symbols or languages while non-symbolic AI is not, there are exceptions. Instead, they essentially represent two ways of learning or gaining intelligence/knowledge: deduction and induction. Symbolic AI features deduction (or deduction reasoning), in which we make inferences based on widely accepted factors or premises, whereas non-symbolic AI boils down to induction (or induction reasoning), in which we extract general knowledge from observations on specific cases. From another perspective, deduction goes from general to special, whereas induction moves from special to general. As a result, symbolic AI features the use of reasoning, usually performed with languages that humans could understand, while non-symbolic AI, e.g., machine learning, is characterized by learning from or via data.

This book is mostly devoted to machine learning, which is predominant in contemporary AI studies. This arrangement was made considering the fact that many people including AI researchers use AI and machine learning interchangeably these days. Despite this fact, it is still worthwhile to mention that symbolic AI might not disappear or go out of date. In fact, in some people's opinions, they have more likely dissolved into our lives. Symbolic AI, which is called "Good Old-Fashioned Artificial Intelligence (GOFAI)", is believed by some other people to be the classical and most successful AI approach till now. AI techniques or efforts along this line make computers more 'intelligent' by using logic, e.g., mathematically provable logical methods, to manipulate 'terms' that were specific to the target task. Humans could define 'rules' for the concrete manipulation of such 'terms' and create rule-based systems. Thus, computers and smartphones that can fulfill tasks based on pre-defined rules built on reasoning like "if statements" can also be viewed as symbolic AI, though we no longer view them as AI in most of the modern AI contexts.

In machine learning, it is common to classify algorithms into supervised learning and unsupervised learning. In addition, semi-supervised learning, as something between and still distinctly different from supervised and unsupervised learning, has also caught lots of attention, especially in recent years. Reinforcement learning is usually deemed as another major category of machine learning in addition to supervised, unsupervised, and semi-supervised learning. This viewpoint is made based on the thought that supervised learning requires a labeled dataset for training, unsupervised learning identifies hidden data patterns from an unlabeled dataset, while reinforcement learning does not require data as it learns by interacting with the environment - generating data in the learning process. Notwithstanding, reinforcement learning can be understood using the frameworks of supervised and unsupervised learning in some way, which makes some people view it as a special case of supervised or unsupervised learning, depending on how labels are defined in the consideration of the rewards. This book primarily covers supervised learning, unsupervised learning, and reinforcement learning.

Another way of classifying machine learning, which is based on the major characteristics or underlying mechanisms, groups machine learning algorithms into Symbolism, Connectionism, Bayesianism, Evolutionism, and Analogism. This classification method is much less common. However, some terms from it such as connectionism may frequently appear in
the context of AI and may confuse people without prior knowledge. Symbolism (investigated by symbolists) here includes symbolic AI in a broad sense. But within machine learning, the narrow definition of it refers to methods characterized by the use of symbols and some types of logic reasoning processes together with learning from data. Examples include decision trees and random forests. Connectionists or Neuroscientists create models based on the brain and thus employ artificial neural networks and their variations including the most recent deep learning, which is a rebrand of the "machine learning with deep neural networks". Bayesians, or more broadly as Bayesian methods, treat machine learning as a form of probabilistic inference. Examples of this category include naive Bayesian, Bayesian Network, Hidden Markov Chains, and graphical models. Evolutionaries or Biologists use genetic algorithms and evolutionary programming, respectively. It is worthwhile to mention that statistical (machine) learning usually represents a much broader concept, in which statistics is used to reinterpret the most popular machine learning algorithms. Thus, it overlaps with machine learning and is distinct from the above Bayesian methods (as a subcategory of machine learning). The use of genetic algorithms appears to be more frequently discussed in the area of optimization than in AI. Analogizers or Psychologists fulfill machine learning tasks based on the similarity between samples or groups. Machine learning models that Analogizers frequently use are K-Nearest Neighbor algorithms, SVMs, and unsupervised machine learning methods.